Problem

When a model group has very few training samples (e.g. <200), the Optuna tuner can select num_leaves values in the thousands. With ~101 training samples and num_leaves=1514, LightGBM effectively memorises the training folds — CV MAPE looks fine because the model can overfit each fold's tiny training set, but out-of-sample performance collapses.

Concrete example from Philadelphia AVM work: residential_mf_large has ~101 training sales. The tuner found num_leaves=1514, which degraded the ratio-study COD from ~40 to ~55 (IAAO standard ≤ 20 for residential). Manually fixing num_leaves=15 recovered COD to ~40.

The same issue can affect min_data_in_leaf: with a range of [20, 500], the upper bound can exceed the entire training fold size, making all splits illegal on the validation side.

Fix

Before constructing the Optuna search space, compute the approximate training-fold size:

n_train_per_fold = int(len(X) * (n_splits - 1) / n_splits)
max_num_leaves = max(8, min(2048, n_train_per_fold // 4))
max_min_data_in_leaf = max(2, min(500, n_train_per_fold // 4))

The // 4 rule means each leaf covers at least ~4 samples on average — a conservative but reasonable floor for a regression tree. Both upper bounds are clamped to the original maximums (2048 and 500), so large datasets are unaffected.

A diagnostic print is emitted under verbose=True when the cap takes effect.

Behaviour at key dataset sizes

For n ≥ ~2560 the num_leaves cap has no effect (reaches the original 2048 ceiling). For n ≥ ~2000 the min_data_in_leaf cap reaches 500 (original ceiling).

Test plan

• Verify existing tests pass (no breakage on normal-sized datasets)
• Smoke-test with a thin group (n < 200): confirm num_leaves in saved params is ≤ n // 5

🤖 Generated with Claude Code